This invention relates to neuronal network simulation and in particular to the use of simulated neuronal networks in computerized apparatus called "automata" for performing basic intellectual and physical tasks. Such tasks may include as an example the recognizing, discriminating, and sorting of various objects or inputs.
The simulated networks and the automata of the present invention are distinguished by their ability to learn as opposed to the mere training of sensorimotor components. During use they develop or improve the criteria by which they recognize and discriminate between input signals. They are capable of categorization, association, and generalization and capable of adaptive behavior based on these abilities. Thus, in use the networks and the automata of which they are a part do not require pre-programming that anticipates all possible variants of the input data they will receive, nor do they have to be pre-programmed with information anticipating the relation of the input data to the output operations of the automaton.
The foregoing features are observed in natural creatures and it has long been the goal to develop neural network simulations that would exhibit them. However, nervous system function is not currently accessible to detailed experimental analysis at the level of adaptive behavior. Prior attempts to simulate nervous system function have relied upon analogy with certain features found in natural neural systems but have been limited in their success. To that end there has been extensive study of the physical characteristics of neural networks in organisms. At present two things are undeniably clear about such systems. First, the physical characteristics of the naturally occurring systems (e.g. the neurons or synaptic junctions) are extremely complex and the number of parameters that are necessary completely to describe such a system is vast. The selection therefore of a group of characteristics that might enable operation of an artificial neural system on a useful level is an extremely complex problem that could hardly be carried out without some kind of automated method. Second, the sheer number of components in any animal is huge compared even to the number of components that are available with the largest of present day computers.
Nature therefore provides examples that display a level of performance that would be desireable in a computerized automaton, but also offers an overabundance of possibilities in how this may be effected and no guarantee that with present hardware it is even possible that an activity of interest can be simulated in a useful manner.
For example, in a preferred embodiment of the invention to be described below, a total of 153,252 simulated synaptic connections are made among 5,747 simulated neurons of 62 different types. In an alternative embodiment of the visual system, also described below, there are 8,521,728 synaptic connections among 222,208 simulated neurons, for an average of 38 connections per unit. In contrast, it is estimated that the human brain has 10.sup.10 neurons and 10.sup.15 synapses, with an average density of 120,000 neurons/mm.sup.3. The density of synapses is on the order of 4.times.10.sup.8 per mm.sup.3, for an average of approximately 4000 synapses per neuron.
It has been surprisingly discovered that despite the relative paucity of connections received by units in the simulation, which undoubtedly reduces the variety and subtlety of their responses, if a careful selection of characteristics is made, sufficient complexity remains to generate useful automata capable of learning and executing tasks of interest.
Others have suggested ways to model integrated cortical action. There has been proposed a hierarchical model in which the visual cortex computes a series of successively abstracted "sketches" of the visual scene. This model, unlike the present invention, is aimed at producing a symbolic description of objects in a scene, and does not incorporate means for categorizing objects or responding to them. Connectionist models for cortical function have also been proposed which incorporate simplified abstract neurons connected to form networks. Systems based on these models have been used to accomplish a number of tasks, including recognition of shapes, pronunciation of written texts, and evaluation of bank loan applications. Most such systems incorporate a "learning algorithm", which adjust the connections of the network for optimal performance based on the presentation of a predetermined set of correct stimulus-response pairs.
A model of sensorimotor coordination has been reported that claims an attempt to replicate real neuronal structures and to utilize neural maps in sensorimotor coordination. However, the model has only limited utility: It is not capable of categorization of the incoming data but merely permits visual signals to drive the position of an arm after training of the system.
Another neurally based model for nervous system function is primarily concerned with visual pattern recognition. When a new stimulus is presented to that model, it searches sequentially in its memory for a recognition template that matches the stimulus; if such a match is not found, the system is able to create a new template which then becomes available for matching in subsequent searches. The present invention, on the other hand, relies upon selection among preexisting, variant recognizing elements to provide responses to novel stimuli. The concept of reentry, used in the present invention to integrate the responses of multiple sensory modalities, is also lacking in the visual pattern recognition model.
The present inventors have also described predecessors of the present invention. The present invention differs from its predecessors by, inter alia, its ability to interact with the environment through motor output, which enables responses that affect sensory input. This feature is termed "reentry". The responses have degrees of adaptive value leading to more complicated behavioral sequences and the possibility of learning. Such learning is accomplished by selection, operating through a new synaptic change rule.